A sparse resultant based method for efficient minimal solvers

Many computer vision applications require robust and efficient estimation of camera geometry. The robust estimation is usually based on solving camera geometry problems from a minimal number of input data measurements, i.e. solving minimal problems in a RANSAC framework. Minimal problems often result in complex systems of polynomial equations. Many state-of-the-art efficient polynomial solvers to these problems are based on Gr\"obner bases and the action-matrix method that has been automatized and highly optimized in recent years. In this paper we study an alternative algebraic method for solving systems of polynomial equations, i.e., the sparse resultant-based method and propose a novel approach to convert the resultant constraint to an eigenvalue problem. This technique can significantly improve the efficiency and stability of existing resultant-based solvers. We applied our new resultant-based method to a large variety of computer vision problems and show that for most of the considered problems, the new method leads to solvers that are the same size as the the best available Gr\"obner basis solvers and of similar accuracy. For some problems the new sparse-resultant based method leads to even smaller and more stable solvers than the state-of-the-art Gr\"obner basis solvers. Our new method can be fully automatized and incorporated into existing tools for automatic generation of efficient polynomial solvers and as such it represents a competitive alternative to popular Gr\"obner basis methods for minimal problems in computer vision

Sparse resultant based minimal solvers in computer vision and their connection with the action matrix

Many computer vision applications require robust and efficient estimation of camera geometry from a minimal number of input data measurements, i.e., solving minimal problems in a RANSAC framework. Minimal problems are usually formulated as complex systems of sparse polynomials. The systems usually are overdetermined and consist of polynomials with algebraically constrained coefficients. Most state-of-the-art efficient polynomial solvers are based on the action matrix method that has been automated and highly optimized in recent years. On the other hand, the alternative theory of sparse resultants and Newton polytopes has been less successful for generating efficient solvers, primarily because the polytopes do not respect the constraints on the coefficients. Therefore, in this paper, we propose a simple iterative scheme to test various subsets of the Newton polytopes and search for the most efficient solver. Moreover, we propose to use an extra polynomial with a special form to further improve the solver efficiency via a Schur complement computation. We show that for some camera geometry problems our extra polynomial-based method leads to smaller and more stable solvers than the state-of-the-art Grobner basis-based solvers. The proposed method can be fully automated and incorporated into existing tools for automatic generation of efficient polynomial solvers. It provides a competitive alternative to popular Grobner basis-based methods for minimal problems in computer vision. We also study the conditions under which the minimal solvers generated by the state-of-the-art action matrix-based methods and the proposed extra polynomial resultant-based method, are equivalent. Specifically we consider a step-by-step comparison between the approaches based on the action matrix and the sparse resultant, followed by a set of substitutions, which would lead to equivalent minimal solvers.Comment: arXiv admin note: text overlap with arXiv:1912.1026

Partially calibrated semi-generalized pose from hybrid point correspondences

In this paper we study the problem of estimating the semi-generalized pose of a partially calibrated camera, i.e., the pose of a perspective camera with unknown focal length w.r.t. a generalized camera, from a hybrid set of 2D-2D and 2D-3D point correspondences. We study all possible camera configurations within the generalized camera system. To derive practical solvers to previously unsolved challenging configurations, we test different parameterizations as well as different solving strategies based on the state-of-the-art methods for generating efficient polynomial solvers. We evaluate the three most promising solvers, i.e., the H51f solver with five 2D-2D correspondences and one 2D-3D correspondence viewed by the same camera inside generalized camera, the H32f solver with three 2D-2D and two 2D-3D correspondences, and the H13f solver with one 2D-2D and three 2D-3D correspondences, on synthetic and real data. We show that in the presence of noise in the 3D points these solvers provide better estimates than the corresponding absolute pose solvers

Calibrated and Partially Calibrated Semi-Generalized Homographies

In this paper, we propose the first minimal solutions for estimating the semi-generalized homography given a perspective and a generalized camera. The proposed solvers use five 2D-2D image point correspondences induced by a scene plane. One of them assumes the perspective camera to be fully calibrated, while the other solver estimates the unknown focal length together with the absolute pose parameters. This setup is particularly important in structure-from-motion and image-based localization pipelines, where a new camera is localized in each step with respect to a set of known cameras and 2D-3D correspondences might not be available. As a consequence of a clever parametrization and the elimination ideal method, our approach only needs to solve a univariate polynomial of degree five or three. The proposed solvers are stable and efficient as demonstrated by a number of synthetic and real-world experiments

A Novel Application of Polynomial Solvers in mmWave Analog Radio Beamforming

Beamforming is a signal processing technique where an array of antenna elements can be steered to transmit and receive radio signals in a specific direction. The usage of millimeter wave (mmWave) frequencies and multiple input multiple output (MIMO) beamforming are considered as the key innovations of 5th Generation (5G) and beyond communication systems. The technique initially performs a beam alignment procedure, followed by data transfer in the aligned directions between the transmitter and the receiver. Traditionally, beam alignment involves periodical and exhaustive beam sweeping at both transmitter and the receiver, which is a slow process causing extra communication overhead with MIMO and massive MIMO radio units. In applications such as beam tracking, angular velocity, beam steering etc., the beam alignment procedure is optimized by estimating the beam directions using first order polynomial approximations. Recent learning-based SOTA strategies for fast mmWave beam alignment also require exploration over exhaustive beam pairs during the training procedure, causing overhead to learning strategies for higher antenna configurations. In this work, we first optimize the beam alignment cost functions e.g. the data rate, to reduce the beam sweeping overhead by applying polynomial approximations of its partial derivatives which can then be solved as a system of polynomial equations using well-known tools from algebraic geometry. At this point, a question arises: 'what is a good polynomial approximation?' In this work, we attempt to obtain a 'good polynomial approximation'. Preliminary experiments indicate that our estimated polynomial approximations attain a so-called sweet-spot in terms of the solver speed and accuracy, when evaluated on test beamforming problems.Comment: Accepted for publication in the SIGSAM's ACM Communications in Computer Algebra, as an extended abstrac

Sparse resultant-based methods with their applications to generalized cameras

Abstract This thesis studies sparse resultants for solving polynomial systems with a view towards camera geometry problems in computer vision. These problems are typically modeled as polynomial systems, parameterized by minimal data samples, and known as minimal problems in computer vision. These polynomial systems present a challenging case as they tend to consist of sparse polynomials with non-generic coefficients. Further, for robust estimation they have to be solved within a RANSAC-like scheme, giving rise to an offline + online strategy. The offline stage has to symbolically analyze the polynomial system, and generate a fast and accurate online stage, i.e. the minimal solver. The most important contribution of this thesis is in algorithms for the offline stage using the sparse resultants via Sylvester-style matrix constructions. Sylvester-style matrices are attractive for offline manipulation as the matrix entries are either 0 or one of the polynomial coefficients. Two algorithms fall in this category: one constructs a sparse resultant matrix by hiding one of the variables of the given polynomial system, while the other algorithm adds an extra polynomial with a special form involving an extra variable and then hides the new variable The two algorithms are automated, and available as open source software. They lead to minimal solvers at the two ends of a spectrum in terms of the number of numerical matrix operations needed as well as the solver stability. One crucial question still remains unanswered: “for a given polynomial system, what is the smallest/fastest solver?” To this end, the thesis demonstrates that it is important to generate minimal solvers using both the proposed resultant-based methods as well as the SOTA action matrix-based methods. The thesis also observes a connection between a convex polytope based resultant matrix construction and the Gröbner basis-based action matrix algorithm, which is investigated, along with an intuitive explanation of the role played by convex geometry in basis selection for resultant-based methods. Lastly, the proposed resultant-based methods are applied to the semi-generalized pose estimation problem and novel minimal solvers are presented for two scenarios: a planar scene and a hybrid set of 2D-2D and 2D-3D point correspondences. The proposed minimal solvers are extensively tested on synthetic and real scenes.Tiivistelmä Tässä väitöskirjassa tutkitaan harvoja resultantteja polynomijärjestelmien ratkaisemiseksi tietokonenäön kamerageometriaongelmien näkökulmasta. Nämä ongelmat mallinnetaan tyypillisesti polynomijärjestelmiksi, parametroidaan minimaalisilla datanäytteillä, ja niistä käytetään nimitystä minimaaliset ongelmat tietokonenäössä. Nämä polynomijärjestelmät edustavat haastavaa tapausta kaikkien ratkaisujen laskemisen kontekstissa, koska ne koostuvat yleensä harvoista polynomeista, joilla on ei-geneeriset kertoimet. Lisäksi luotettavaa estimointia varten ne on ratkaistava RANSAC:n kaltaisessa järjestelyssä, mikä johtaa offline + online -strategiaan. Offline-vaiheen täytyy symbolisesti analysoida polynomijärjestelmää ja luoda nopea ja tarkka online-vaihe, eli minimiratkaisija. Tämän väitöskirjan tärkein kontribuutio on offline-vaiheen algoritmeissa, joissa käytetään harvaa resultanttia Sylvester-tyylisten matriisirakenteiden kautta. Sylvester-tyyliset matriisit ovat houkuttelevia offline-käsittelyyn, koska yksittäinen matriisialkio on joko nolla tai jokin polynomikertoimista. Kaksi algoritmia kuuluu tähän luokkaan: toinen muodostaa harvan resultanttimatriisin piilottamalla yhden annetun polynomijärjestelmän muuttujista, kun taas toinen algoritmi lisää ylimääräisen polynomin, jolla on erityinen muoto sisältäen ylimääräisen muuttujan, ja piilottaa sitten kyseisen uuden muuttujan. Molemmat algoritmit ovat automatisoituja ja saatavilla avoimen lähdekoodin ohjelmistoina. Nämä kaksi ehdotettua algoritmia johtavat minimaalisiin ratkaisijoihin spektrin kahdessa ääripäässä tarvittavien numeeristen matriisioperaatioiden lukumäärän sekä ratkaisijan stabiilisuuden kannalta. Yksi ratkaiseva kysymys on edelleen vaille vastausta: mikä on annetulle polynomijärjestelmälle pienin/nopein ratkaisija? Tätä tarkoitusta varten väitöskirja osoittaa, että on tärkeää generoida minimiratkaisijoita käyttäen sekä ehdotettuja resultanttipohjaisia menetelmiä että alan viimeisimpiä toimintamatriisipohjaisia menetelmiä. Väitöskirjassa tehdään myös havainto konveksiin polytooppiin perustuvan resultanttimatriisikonstruktion ja Gröbnerin kantapohjaisen toimintamatriisialgoritmin välisestä yhteydestä, jota tutkitaan yhdessä intuitiivinen selityksen kanssa konveksin geometrian roolista resultanttipohjaisten menetelmien kantavalinnassa. Tässä väitöskirjassa sovelletaan myös ehdotettuja resultanttipohjaisia menetelmiä puoliyleistettyyn asennon estimointiongelmaan ja esitetään uusia minimiratkaisijoita kahdelle skenaariolle, ts. tasomainen näkymä ja hybridijoukko 2D-2D ja 2D-3D pistevastaavuuksia. Ehdotetut minimiratkaisijat on testattu laajasti synteettisillä ja todellisilla näkymillä

A sparse resultant based method for efficient minimal solvers

Abstract Many computer vision applications require robust and efficient estimation of camera geometry. The robust estimation is usually based on solving camera geometry problems from a minimal number of input data measurements, i.e. solving minimal problems in a RANSAC framework. Minimal problems often result in complex systems of polynomial equations. Many state-of-the-art efficient polynomial solvers to these problems are based on Gröbner basis and the action-matrix method that has been automatized and highly optimized in recent years. In this paper we study an alternative algebraic method for solving systems of polynomial equations, i.e., the sparse resultant-based method and propose a novel approach to convert the resultant constraint to an eigenvalue problem. This technique can significantly improve the efficiency and stability of existing resultant-based solvers. We applied our new resultant-based method to a large variety of computer vision problems and show that for most of the considered problems, the new method leads to solvers that are the same size as the the best available Gröbner basis solvers and of similar accuracy. For some problems the new sparse-resultant based method leads to even smaller and more stable solvers than the state-of-the-art Gröbner basis solvers. Our new method can be fully automatized and incorporated into existing tools for automatic generation of efficient polynomial solvers and as such it represents a competitive alternative to popular Gröbner basis methods for minimal problems in computer vision

Computing stable resultant-based minimal solvers by hiding a variable

Abstract Many computer vision applications require robust and efficient estimation of camera geometry. The robust estimation is usually based on solving camera geometry problems from a minimal number of input data measurements, i.e., solving minimal problems, in a RANSAC-style framework. Minimal problems often result in complex systems of polynomial equations. The existing state-of-the-art methods for solving such systems are either based on Gröbner bases and the action matrix method, which have been extensively studied and optimized in the recent years or recently proposed approach based on a resultant computation using an extra variable. In this paper, we study an interesting alternative resultant-based method for solving sparse systems of polynomial equations by hiding one variable. This approach results in a larger eigenvalue problem than the action matrix and extra variable resultant-based methods; however, it does not need to compute an inverse or elimination of large matrices that may be numerically unstable. The proposed approach includes several improvements to the standard sparse resultant algorithms, which significantly improves the efficiency and stability of the hidden variable resultant-based solvers as we demonstrate on several interesting computer vision problems. We show that for the studied problems, our sparse resultant based approach leads to more stable solvers than the state-of-the-art Gröbner basis as well as existing resultant-based solvers, especially in close to critical configurations. Our new method can be fully automated and incorporated into existing tools for the automatic generation of efficient minimal solvers

Partially calibrated semi-generalized pose from hybrid point correspondences

Abstract We study the problem of estimating the semi-generalized pose of a partially calibrated camera, i.e., the pose of a perspective camera with unknown focal length w.r.t. a generalized camera, from a hybrid set of 2D-2D and 2D-3D point correspondences. We study all possible camera configurations within the generalized camera system. To derive practical solvers to previously unsolved challenging configurations, we test different parameterizations as well as different solving strategies based on state-of-the-art methods for generating efficient polynomial solvers. We evaluate the three most promising solvers, i.e., the H51f solver with five 2D-2D correspondences and one 2D-3D match viewed by the same camera inside the generalized camera, the H32f solver with three 2D-2D and two 2D-3D correspondences, and the H13f solver with one 2D-2D and three 2D-3D matches, on synthetic and real data. We show that in the presence of noise in the 3D points these solvers provide better estimates than the corresponding absolute pose solvers